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Pre-face 


The  purpose  o-f  this  study  was  to  begin  the  process  o-f 
demonstrating  the  utility  o-f  the  TSAR  model  to  the  Air  Force 
manpower  requirements  analysts.  In  particular,  the  study 
aimed  to  show  whether  or  not  TSAR  outputs  could  match  those 
o-f  the  LCOM  model  ,  which  has  been  a  standard  -for  manpower 
requirements  -forecasts  for  many  years. 

Results  of  the  comparison  were  inconclusive  from  a 
statistical  standpoint,  due  primarily  to  incomplete  coordi¬ 
nation  of  the  two  model  inputs.  Man-hour  outputs  from  TSAR 
showed  consistently  lower  totals  than  did  LCOM,  at  the  same 
sortie  tasking.  At  the  same  time,  TSAR  was  able  to  simulate 
completion  of  a  greater  percentage  of  the  assigned  sorties. 
Qualitative  differences  between  TSAR  and  LCOM  were  noted. 
TSAR  has  generally  poorer  data  input  and  output,  due  to  the 
lack  of  p re-  and  post-processors. 

In  performing  the  experimentation  and  writing  this 
thesis,  I  am  indebted  to  the  support  of  many  others.  I 
would  especially  like  to  thank  my  faculty  advisor,  Lt  Cal 
John  Halliday  for  his  tireless  support  and  encouragement.  I 
also  wish  to  thank  Mr.  Richard  Cronk  and  Mr.  Elliot  Wunsh  of 
Aeronautical  Systems  Division  for  their  patience  and  guid¬ 
ance  in  teaching  LCCM  to  me.  I  most  especially  want  to 
thank  my  wife  Irene  far  being  there  when  I  needed  her. 


David  R.  Ncfcle 
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Abstract 


I 

A  comparison  was  made  between  the  outputs  c-f  TSAR  and 
LCQM.  Each  model  was  run  with  a  F-16  database  representing 
peacetime  maintenance  procedures  at  a  single  base.  Since 
TSAR  is  a  wartime  model,  modifications  to  the  TSAR  database 
were  made  in  an  attempt  to  make  the  models  compatible.  Sim¬ 
ulation  runs  were  made  using  three  -flying  schedules,  repre¬ 
senting  1.0,  2.0,  and  3.0  sortie  rate  taskings.  Sortie 
production  and  man-hour  outputs  from  like  TSAR  and  LCCtl  runs 
were  compared.  Statistical  comparison  of  the  outputs  shewed 
TSAR  sortie  production  figures  and  man-hour  outputs  varied 
si gni f i cantl y  from  those  given  by  LCOM.  Results  were  biased 
by  the  lack  of  completely  compatible  databases,  but  there 
was  some  evidence  to  suggest  that  a  fairly  constant  factor 
represented  the  difference  in  outputs.  In  the  course  of  the 
study,  qualitative  differences  in  the  models  were  noted. 
These  diffsrencss  impacted  the  comparative  ease  of  use  of 
the  models  and  their  suitability  to  specific  appl i cat i ons. 
TSAR  is  generally  more  difficult  to  use,  in  both  input  and 
output  preparation  and  usage,  but  has  greater  pewer  across  a 
wide  range  of  options.  Development  of  TEAR  pre-  and  post¬ 
processors  is  suggested  as  a  way  to  improve  ease  of  input 


COMPARISON  OF  THE  TSAR  MODEL  TO  THE  LCOM  MODEL 


I .  Introduction 

Issue 

Does  the  Theater  Simulation  of  Airbase  Resources  (TSAR) 
model  give  forecasts  of  aircraft  maintenance  requirements 
that  are  comparable  to  the  "standard"  forecasts  provided  by 
the  Logistics  Composite  (LCOM)  model?  This  question  is  of 
interest  to  analysts  at  Headquarters ,  United  States  Air 
Force  (HQ  USAF) ;  the  Air  Force  Manpower  and  Personnel  Center 
(AFMPC);  Headquarters ,  Air  Force  Logistics  Command  (HQ 
AFLC) ;  the  Aeronautical  Systems  Division  (ASD)  and  at  ether 
locations  around  the  Air  Force  (AF)  (15). 

Background.  The  military  uses  a  variety  of  computer 
simulation  models  to  project  manpower,  spars  part  and  other 
resource  needs  (2:7-3;  17:2).  LCOM  and  TSAR  are  examples  of 

military  simulation  models  designed  to  forecast  resource 
needs . 

Both  LCOM  and  TSAR  are  Monte  Carlo,  discrete-event  sim¬ 
ulation  models  (1:  7,44;  6:  Sec  2;  8:  1)  .  Bath  have  as  a 

basic  abjective  investigation  into  how  the  i nterrel ati ons  of 
numerous  resources  impact  the  operation  of  a  weapon  system 
(6:  Sec  1,  1;  8:1).  LCOM  simulates  and  analyzes  ".  .  .  sup¬ 

port  resources  usage  (personnel,  equipment,  spare  parts, 


stc.)  and  the  impact  of  their  availability  on  the 
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operational  status  of  the  weapon  system  ..."  (6:  Sec  2,1). 

Likewise  TSAR  "...  analyses  the  i nterrel at i cns  among 
available  resources  .  .  to  generate  aircraft  sorties  ..." 

(8:1).  In  other  words,  the  models  simulate  the  activities 
associated  with  the  operation  of  a  weapon  system.  For  ex¬ 
ample,  TSAR  considers  11  different  resource  types  associated 
with  generating  aircraft  sorties  (1)  aircraft,  (2)  aircrews, 
(3)  ground  personnel,  (4)  support  equipment  (A6E) ,  (5)  air¬ 

craft  parts,  (6)  aircraft  shelters,  (7)  munitions,  (S) 
tanks,  racks,  adaptors  and  pylons  (TRAP),  (9)  petroleum, 
oils  and  lubricants  (POL),  (13)  building  materials,  and  (11) 
airbase  facilities  (8:1).  LCOM  simulates  a  similar  but 
slightly  smaller  list  of  resources  (6:  Sec  2). 

The  models  process  a  series  of  user  identified  tasks 
that  correspond  to  the  actual  requirements  of  weapon  system 
cper at i on .  Seme  of  these  tasks  may  be  probabalistic,  that 
is,  occurring  only  as  the  result  of  some  random  event.  Sim¬ 
ulation  models,  in  general ,  handle  probabalistic  events  by 
drawing  a  random  number  from  a  random  number  generator,  and 
modifying  it  tc  fit  the  desired  probability  distribution  (1: 
256).  The  size  of  the  resulting  number  is  compared  tc  the 
stated  orobability  of  an  event  occurring.  The  appropriate 
action,  either  to  process  cr  to  skip  the  event,  is  taken,  as 
a  -esult  of  that  comparison.  Both  LCOM  and  TSAR  use  random 
numbers,  although  their  generation  and  specific  uses  vary 
•6:  9'.  Other  tasks  may  be  designated  so  as  to  oe  processed 
-oucinel/  '1037.  prooaoility  of  occurrence)  at  seme  steo  in 


the  simulation  cycle.  The  user  may  elect  to  change  the  de¬ 
fined  tasks,  in  an  attempt  to  identify  the  costs  and  bene¬ 
fits  of  alternative  modes  of  operation. 

The  user  may  also  specify  that  certain  resources  are 
necessary  for  the  completion  of  a  task.  Those  resources  may 
be  consumed  by  the  task  or  they  may  be  returned  to  a  pool 
for  future  use,  after  completion  of  the  assigned  task.  The 
user  may  vary  the  availability  of  resources  in  order  to  in¬ 
vestigate  the  sensitivity  of  the  system  to  that  particular 
resource  or  combination  of  resources. 

The  differences  between  LCOM  and  TSAR  revolve  around 
the  advanced  state  of  development  of  LCOM  (as  compared  to 
TSAR)  and  the  expanded  list  of  activities  and  resources  that 
TSAR  can  simulate. 

LCQM-Unique  Capabi 1 i ties.  LCOM  enjoys  wide  use 
within  the  Air  Force  and  has  been  adapted  to  perform  a  num¬ 
ber  of  different  functions  (6:  Sec  2  ,1-2).  Through  many 
years  of  development  <4) ,  LCOM  has  gained  a  variety  of  func¬ 
tions  dedicated  to  simplifying  input  and  combining  output 
into  more  useable  forms  <6:  Sec  2,  2-4).  TSAR,  being  a  much 
newer  creation,  does  not  have  as  great  a  library  of  support 
functions.  For  instance,  LCOM  uses  a  pre-processor .  This 
allows  the  user  to  format  input  data  in  a  highly  understand¬ 
able  form.  The  pre-processor  then  takes  the  data  and  pre¬ 
pares  it  for  use  in  the  simulation.  The  output  of  the  pre¬ 
processor  is  much  more  complex  and  difficult  for  the  user  to 


comprehend,  but  is  in  the  form  necessary  for  submission  to 


the  main  module  (simulation) 


TSAR  does  not  yet  have  a  pre¬ 


processor,  although  development  is  underway  (4),  therefore 
it  suffers  in  comparison  to  LCQM  in  ease  of  data  input. 

TSAR-Uni que  Capabi 1 i ti es.  TSAR  has  the  capability 
to  simulate  a  much  broader  set  of  options  than  does  LCQM  (8: 
2).  for  instance,  TSAR  users  can  simulate  the  operation  of 
a  single  airbase,  several  independent  airbases,  or  a  set  of 
interdependent  airbases.  LCOM  is  presently  limited  to  sin¬ 
gle  airbase  operations.  TSAR  can  also  model  the  effects  of 
conventional  or  chemical  airbase  attacks,  the  effects  of 
having  personnel  operate  in  individual  chemical  protection 
equipment  (IF'E),  and  the  efforts  to  reconstitute  the  aii — 
base  following  an  attack.  LCOM  has  none  of  these  options. 
TSAR  gives  the  user  the  ability  to  model  the  activities 
within  an  entire  theatre  of  operations,  under  wartime  condi¬ 
tions.  Thus  activities  such  as  lateral  support  between 
bases  can  be  assessed.  Within  each  base,  TSAR  provides  a 
wider  span  of  acti vitas,  thereby  allowing  the  user  to  assess 
the  added  capabilites  or  restrictions  imposed  by  such  sup¬ 
port  functions  as  civil  engineering  and  air  traffic  control . 

Other  Pi f f erences .  Since  LCOM  has  been  used  for 
such  a  long  time,  it  has  been  repeatedly  validated,  that  is, 
had  its  results  compared  against  real  world  airbase  opera¬ 
tions  (4' .  Because  Qf  its  long  use,  flexibility  and  demon¬ 
strated  validity,  LCOM  has  become  institutionalized  as  an 
accepted  resources  determination  tool.  LCOM  has  some  limit¬ 
ations,  however.  First,  LCOM  is  written  in  SIMSCRIF’T  and 


therefore  requires  a  computer  system  with  a  SIMSCRIPT  com¬ 
piler  (12:10).  This  can  greatly  reduce  the  choice  an  agency 
has  in  buying  a  computer  system,  if  they  intend  to  use  LCOM 
to  do  requirements  forecasting  or  other  capability  assess¬ 
ments  (15).  Also,  LCOM  model  runs  take  a  very  long  time  to 
complete,  often  forcing  the  analyst  to  do  runs  late  at  night 
when  computer  utilization  is  low  (4).  For  these  reasons, 
and  to  overcome  LCOM's  limitations  in  modeling  important 
wartime  concepts,  the  TSAR  model  was  developed,  in  lieu  of 
modifying  LCOM  to  incorporate  multi-base  and  combat 
scenarios. 

TSAR  has  also  undergone  successful  validation,  but  not 
to  the  extent  LCOM  has.  TSAR  is  written  in  FORTRAN  (10:v), 
which  is  one  of  the  most  common  computer  languages  (14). 
Availability  of  a  FORTRAN  compiler  is  rarely  a  factor  in  the 
choice  of  a  computer  system  (14).  FORTRAN  is  also  a  more 
efficient  language,  from  the  point  of  view  of  computer 
speed,  and  therefore,  TSAR  simulations  are  much  faster  than 
similar  LCOM  runs  (15).  On  the  other  hand,  TSAR  can  only 
model  a  2-shift,  wartime  maintenance  arrangement  as  opposed 
to  the  3-shift  arrangement  common  in  peace-time  operations 
(15).  While  TSAR  is  gaining  acceptance  as  a  useful  model 
within  those  areas  where  it  supplements  LCOM  (21),  it  has 
not  been  i nsti tuti onal i zed  within  the  AF,  i.e.,  there  is  no 
single  agency  responsible  for  TSAR  development  and  mainte¬ 
nance.  LCOM,  on  the  other  hand,  is  "owned  and  operated"  by 
the  AF  Manpower  community  (15). 


Rati onal e  for  Model  Comparisons.  Since  LCQM  has  limit¬ 
ations  in  what  situations  it  can  model  (10:2),  analysts  are 
searching  -for  alternatives  to  its  use  (15).  While  TSAR  in¬ 
cludes  many  useful  extensions  that  LCQM  lacks  (8),  analysts 
are  reluctant  to  use  it  exclusively  because  its  ability  to 
match  the  forecasting  ability  and  suitability  of  LCQM,  in 
those  areas  where  the  two  models  duplicate  capability,  has 
yet  to  be  proven  (15).  As  a  result,  analysts  are  faced  with 
the  undesirable  prospect  of  having  to  maintain  both  models 
in  order  to  get  useable  forecasts  across  the  full  range  of 
aircraft  maintenance  and  airbase  activities.  This  is  unde¬ 
sirable  because  both  models  are  large' and  complex,  requiring 
a  lot  of  upkeep.  For  example,  a  simple  TSAR  task  (card  type 
(CT)  #5)  contains  up  to  26  pieces  of  data  describing  the 
type  of  task  to  be  performed,  the  probability  of  its  occur — 
rence,  the  length  of  the  task,  the  number  and  kinds  of  re¬ 
sources  used  for  the  task,  the  location  of  subsequent  and 
parallel  tasks,  the  distributions  of  probabal i sti c  events, 
such  as  task  length,  and  other  data  (9:46).  In  the  TSAR 
database  used  in  this  study,  there  are  972  CT  #5s  (19), 
meaning  that  this  very  simple  database  (15)  has  the  poten¬ 
tial  for  needing  up  to  25,272  (26  X  972)  separate  pieces  of 
data  for  inclusion  in  the  CT  #5s  alone.  Although,  no  single 
CT  #5  is  likely  to  contain  all  26  passible  pieces  of  data, 
the  total  number  of  data  is  quite  large,  especially  consid¬ 
ering  CT  #5s  comprise  only  about  one  half  of  the  total  data¬ 
base  used  in  this  study  (20).  In  addition,  each  model  de- 


mands  separate  training  -for  users  and  different  computer 
capabilities  (10; v).  In  short,  maintaining  both  models  re¬ 
sults  in  a  great  deal  of  duplication,  which  is  costly  and 
inefficient.  If  users  had  proof  that  TSAR's  outputs  were 
comparable  to  LCOM's,  TSAR  could  be  used  exclusively,  in 
most  applications  (IS). 

Problem  Statement 

The  specific  question  this  study  will  address  is:  Does 
TSAR  provide  acceptable  man-hour  usage  and  sortie  production 
forecasts  when  the  daily  sortie  rate  requirement  is  varied 
across  representati ve  levels?  Acceptable  is  operationally 
defined  as  being  statistically  equivalent,  within  a  speci¬ 
fied  confidence  interval,  of  the  forecasts  provided  by  LCOM. 

Rationale  for  Problem  Statement  Limitations.  Bath  the 
LCOM  and  TSAR  models  simulate  the  full  range  of  activity 
within  an  aircraft  maintenance  organization  (8;  10;  12).  In 

addition,  TSAR  has  the  capability  to  model  across  several 
maintenance  organizations  scattered  across  a  theater — wide 
area,  plus  the  ability  to  handle  enemy  actions  (8;  12). 

This  means  each  model  can  take  hundreds  of  inputs,  process 
them  through  innumerable  permutations  of  the  networks  and 
provide  hundreds  of  output  data  (10;  12).  Clearly  it  is 
beyond  the  scope  of  a  single  research  effort  to  evaluate  the 
TSAR  model  across  the  entire  spectrum  of  its  capability. 

The  thrust  of  this  study  is  to  analyze  two  selected  outputs, 
with  limited  inputs,  in  order  to  get  a  representati on  of  how 
closely  TSAR  matches  LCOM.  Man-hour  usage  figures  were 


chosen  because  use  of  the  models  for  manpower  analyses  is 
one  of  their  principal  purposes  (15).  The  sortie  production 
statistic  was  chosen  because  it  directly  reflects  the  level 
of  maintenance  activity  undertaken. 

Research  Objectives 

In  addition  to  the  specific  problem  identified,  this 
study  will  address  the  following  research  questions. 

1.  What  differences  exist  between  LOOM  and  TSAR 
simulations? 

2.  In  what  ways  must  TSAR  be  manipulated  to  give 
acceptable  forecasts? 

3.  If  TSAR  fails  to  provide  acceptable  forecasts,  are 
there  apparent  shortcomings  in  TSAR  that  can  be 
economically  overcome? 

Network  Pi f f erences.  There  are  a  number  of  differences 
in  the  networks  between  LOOM  and  TSAR  (10).  The  thrust  of 
research  question  number  1  is  to  identify  what,  if  any,  of 
these  differences  are  of  consequence  to  the  user,  how  they 
affect  the  use  of  TSAR  and  what,  if  anything  can  be  done  to 
limit  the  impact  of  those  differences. 

Manipulations.  TSAR  has  a  great  capacity  for  the  model 
user  to  tailor  the  simulation  to  suit.  For  instance,  the 
probability  distributions  of  chance  occurences  can  be  speci¬ 
fied  by  the  user  (10).  Question  number  2  deals  with  what 
particular  manipulations  the  user  may  need  to  specify  in 
order  for  TSAR  to  adequately  emulate  LOOM. 

Shortcomings.  Even  if  TSAR  fails  to  give  acceptable 
forecasts  in  its  present  form,  the  economies  acnieved  by 
reducing  the  requirement  to  maintain  both  models  may  make  it 


■feasible  to  modify  TSAR  to  give  acceptable  forecasts.  The 
goal  of  research  question  number  3  is  to  identify  where  TSAR 
falls  short  in  matching  LCOM  outputs,  and  to  explore  possi¬ 
bilities  for  relatively  simple  and  inexpensive  modifications 
to  TSAR  that  would  improve  its  forecasts. 

Ex per i mental  Hypotheses 

The  overall  purpose  of  this  study  is  to  test  the  propo¬ 
sition  that  TSAR  and  LCDM  outputs  are  functionally  the  same. 

Because  of  the  limitations  of  the  study,  the  actual 
proposition  to  be  tested  is: 

TSAR  and  LCQM  man-hour  usage  and  sortie  production 
outputs  are  functionally  the  same,  when  the  target 
daily  sortie  rats  is  varied. 

Scope  of  the  Research 

In  order  to  deal  with  the  complexity  of  the  LCOM  and 
TSAR  models,  limitations  are  imposed  in  each  of  two  areas 
related  to  the  simulations:  inputs  and  outputs. 

Input  Limitations.  The  inputs  to  the  models  will  be 
the  same  for  each  model.  The  input  values  correspond  to  a 
maintenance  data  base  for  the  F-16  tactical  fighter.  The 
only  input  that  will  be  varied  is  the  target  daily  sortie 
rate.  See  appendix  A. 

Output  Li  mi tati ons.  The  only  outputs  of  the  models 
that  will  be  considered  for  comparison  are  the  man-hour 


usage  and  sortie  production  outputs.  See  appendix  B. 


Gummary 

Written  in-formation  about  the  rationale  for  this  study 
is  limited-  Rather  the  bulk  of  the  justification  comes  from 
the  users  <4;  7;  15;  21).  TSAR  provides  important  exten¬ 
sions  to  LCQM  (10:2)  and  these  extensions  are  unlikely  to  be 
matched  by  further  modification  of  the  LCOM  networks  (4). 

In  addition,  TSAR  is  written  in  the  commonly  available 
FORTRAN  general  purpose  computer  language  (10;v)  while  LCOM 
is  written  in  the  harder — to-find  SIMSCRIPT  specialized  sim¬ 
ulation  language  (12:10).  This  means  that  TSAR  is  more 
portable,  since  computer  systems  that  have  FORTRAN  compilers 
are  common.  It  also  makes  suitable  operating  systems  easier 
and  cheaper  to  find  (15).  If  belief  in  TSAR  is  warranted, 
analysts  would  be  well -served  by  embracing  TSAR  as  a 
general-purpose,  requi rements ,  resource  allocation  and 
capability  assessment  model  replacing  LCOM. 

Def i  nj  ti cn  of  Terms 

Ai r  Force  Speci al ty  Code.  An  air  force  specialty  code 
( AFSC)  is  defined  as:  a  numerical  code  designating  the 
career — field  specialty  of  the  individual.  For  the  purposes 
of  this  study,  AFSCs  designate  the  particular  skills  used  as 
manpower  resources  in  the  simulations.  For  example,  AFSC 
30470  represents  a  ground  radio  maintenance  technician. 

Network .  A  network,  is  defined  as:  a  block  of  computer 
cede  in  the  simulation  program  designed  to  represent  a  part¬ 
icular  system  or  sub-system  of  the  actual  process  being  mod¬ 
eled.  Figure  1  depicts  a  flow  diagram  representing  TSAF's 


Figure  1.  TSAR  Ejection  Seat  Repair  (20!  Sec  HI,  49) 


modeling  of  ejection  seat  repair  on  the  F-16.  The  computer 
code  corresponding  to  -figure  1  ia  an  example  o-f  a  network. 

Target  Dai 1 y  Sortie  Rate.  The  target  daily  sortie  rate 
is  operationally  defined  as:  a  desired  level  o-f  aircraft 
utilization  expressed  as  the  number  of  missions  per  day  each 
aircraft  assigned  to  a  unit  is  to  perform.  This  value  is 
used  to  determine  the  total  sortie  demand  per  day,  e.g.. 


<#  of  aircraft  assigned)  X  (target  daily  sortie  rate)  = 

total  sortie  demand 


Ij 
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Justi  f  i  cat  i  on  of  Experimental  Approach 


R 


The  experimental  approach  was  chosen  -for  this  study 
because  .  .no  other  method  can  approach  CtheD  power  o-f 
experimentation  to  determine  causal  relationships  between 
variables"  (11:129).  Further  the  nature  o-f  this  study  makes 
an  experimental  design  easy  to  implement.  Since  the  systems 
under  study  are  models,  manipulation  of  the  variables  is  not 
a  problem.  Finally,  experimentation  allows  control  of  more 
than  one  variable  (11:129).  This  is  an  important  considera¬ 
tion  because  one  of  the  major  objectives  of  the  study  is  to 
determine  if  a  difference  in  target  daily  sortie  rate  has  an 
effect  on  the  correlation  between  LCOM  and  TSAR  (15). 

Ex pi anati on  of  Experimental  Design 

The  experimental  design  used  in  this  study  is  a  random¬ 
ized  block  design.  The  randomized  block  design  allows  for 
the  testing  of  two  independent  variables  at  the  same  time 
(model  type  and  target  daily  sortie  rate)  (1:123).  Not  only 
can  both  independent  variables  be  tested,  but  also  the  indi¬ 
vidual  effects  of  each  variable  can  be  separated  and  con¬ 
trolled  (18:361).  Emory  gives  steps  to  setting  up  a  random¬ 
ized  block  design.  First,  the  independent  variable  cf  pri¬ 
mary  interest  (model  type)  is  broken  into  levels,  usually 
chose  of  interest  to  the  experimenter.  In  this  case,  the 
levels  are  LCOM  and  TSAR.  Then,  the  secondary  independent 
variable  (target  daily  scrte  rate)  is  also  broken  into 
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Figure  2.  Experimental  Design  for  Comparing  LCOM  and  TSAR 


levels.  -These  levels  may  be  those  of  interest  to  the  re¬ 
searcher  or  often  they  reflect  an  aspect  of  the  papulation 
that  is  beyond  the  ability  of  the  experimenter  to  control 
directly.  For  the  purposes  of  this  study,  target  daily 
sortie  rate  (TDSR)  ,  which  is  under  the  control  of  the 
researcher,  will  be  held  at  three  levels:  1.0,  2.0,  and  3.0. 
See  figure  2.  Samples  are  drawn  at  random  from  the  range  of 
the  passible  field  of  study  and  those  samples  are  subjected 
to  the  independent  variables  (1:122-124).  Conclusions  about 
the  effect  of  the  variables  are  drawn  by  comparing  the 
averaged  impact  of  all  the  levels  of  one  independent 
variable,  while  holding  the  other  independent  variable 
constant  (1:124).  If  the  results  of  the  experiment  are 
numerical,  statistical  means  can  be  used  to  make  the 
comparisons  between  and  within  variables  (18:646-654). 
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Input  Vari abl es.  The  primary  independent  variable  in 
the  study  will  be  the  model  used  to  analyze  the  data  base. 
The  model  used  can  consist  of  two  levels  or  treatments: 
Treatment  1  -  LCQM 
Treatment  2  -  TSAR 

The  secondary  independent  varible  will  be  target  daily 
sortie  rate.  Addition  of  this  secondary  independent  vari¬ 
able  will  allow  analysis  o-f  whether  variance  of  the  input 
values  changes  the  degree  to  which  TSAR  outputs  equate  to 
LCQM  outputs  (1:123).  Target  daily  sortie  rate  will  con¬ 
sist  of  three  levels  or  blacks: 

Black  1  -  target  daily  sortie  rate  of  1.0 

Block  2  -  target  daily  sortie  rate  of  2.0 

Black  3  -  target  daily  sortie  rate  of  3.0 

Output  Var i ables.  Two  output  variables  will  be  com¬ 

pared,  man-hour  usage  and  sortie  production.  Man-hour  usage 
outputs  are  broken  down  either  by  AFSC  or  by  the  shop  to 
which  the  personnel  are  assigned,  and  will  reflect  the  num¬ 
ber  of  man-nours  utililized  in  the  networks.  Sortie  produc¬ 
tion  will  be  expressed  as  the  total  number  of  sorties  that 
are  actually  flown  (figure  2). 

Experimental  Control s 

The  papulation  for  this  experiment  is  the  total  number 


of  outputs  that  are  common  to  LCQM  and  TSAR. 


For  purposes 


o f  making  the  study  of  manageable  size,  only  two  of  the  out¬ 
puts  will  be  chosen  as  the  sample.  Since  each  output  exists 
in  each  model,  albeit  in  different  forms,  group  equivalency 
is  not  a  concern.  However,  random  error  will  be  induced 
because  of  the  stochastic  nature  of  the  models.  Stochastic 
models  are  defined  as  those  that  have  random  variables  as 
inputs  (1:10).  This  randomness  means  that  the  models  may 
give  slightly  different  outputs  even  though  the  user-defined 
inputs  are  the  same.  As  suggested  by  Banks  and  Carson,  the 
effect  of  random  error  on  the  accuracy  of  the  values  for 
comparison  will  be  controlled  by  using  the  mean  (average)  of 
the  difference  between  comparable  output  variables  of  both 
models.  The  means  will  be  computed  from  multiple  runs  with 
the  same  input  data  (1:451).  In  other  words,  multiple  runs 
of  each  model  will  be  made  at  each  TDSR,  a  difference  found 
between  randomly  paired  runs  (within  the  same  TDSR)  and  the 
mean  of  the  differences  calculated. 

Ideally,  the  models  would  be  run  with  the  same  random 
number  strings  to  insure  synchronization  of  the  model 
outputs  (1:453).  However,  with  TSAR  and  LCOM  being  so 
different,  this  is  impractical.  To  compensate,  it  would  be 
desirable  to  do  a  larger  number  of  longer  runs  than  would 
otherwise  be  necessary  to  determine  the  output  means. 
However,  the  high  cost  and  time  involved  with  doing  large 
numbers  of  replications  of  long  LCOM  and  TSAR  runs  will  set 
a  practical  limit  on  hew  many  are  actually  done.  A  limit  of 
5-30  day  runs  of  each  model  at  each  TDSR  were  dene,  for 


purposes  of  economy.  Finally,  simulation  models  such  as 
LCOM  and  TSAR  are  normally  allowed  to  "warm-up"  prior  to 
outputs  being  tabulated  in  order  to  avoid  bias  in  the 
outputs  (1:73).  However,  -for  use  of  models  to  simulate 
wartime  conditions,  the  transients  involved  in  the  early 
portion  of  the  simulation  are  of  interest,  as  they  are 
indicative  of  how  the  early  days  of  a  conflict  may  differ 
from  the  "steady-state"  encountered  after  a  period  of  con¬ 
flict  has  elapsed  (15).  Therefore,  the  models  were  run 
without  warmup. 

Criteria  for  Sel ecti on  i nto  Groups 

The  criteria  far  selection  of  the  specific  outputs  to 
be  included  in  this  sample  is  purely  judgmental,  as  opposed 
to  selection  at  random.  Sortie  production  was  chosen  be¬ 
cause  it  is  indicative  of  the  total  maintenance  effort  rep¬ 
resented  by  the  simulation  and  is  the  "bottom-line"  when 
considering  alternative  maintenance  concepts  (7).  Man-ncur 
usage  outputs  were  chosen  because  manpower  forecasting  is  of 
prime  importance  to  a  major  segment  of  potential  users  of 
the  models,  the  AF  Manpower  community.  Excluded  outputs 
were  not  considered  for  inclusion  in  the  sample  for  the  sake 
of  making  the  study  of  manageable  scope.  Since  judgment 
selection  is  a  non-probability  means  of  sampling  (11:280), 
g enar a! i z at i on  of  the  comparison  results  to  the  entire  popu¬ 
lation  of  the  outputs  is  not  appropriate  (11:279). 


Criteria  -for  Analysis 

Hypotheses.  The  general  purpose  of  the  analysis  is  to 
test  the  fallowing  hypotheses: 

H0:  The  outputs  of  the  models  are  the  same 
H«:  The  outputs  of  the  models  are  different 
General  Category  of  Statistical  Test.  Since  the 
randomized  block  design  used  in  this  study  has  only  two 
treatments,  a  paired  difference  test  can  be  used  to  compare 
model  outputs  (18:361).  The  specific  statistical  test  that 
will  be  used  depends  on  the  probability  distribution  of  the 
outputs  (3) .  Parametric  tests  are  preferred  because  they 
generally  have  greater  power  to  identify  differences  (3). 
However,  as  McClave  indicates,  one  of  the  underlying  assump¬ 
tions  in  the  use  of  parametric  statistics  is  that  the  proba¬ 
bility  distribution  of  the  tested  variables  is  approxi matel y 
normal.  If  this  assumption  cannot  be  made,  nonparametric 
comparison  tests  must  be  used  (18:674). 

Determination  of  Sample  Distribution.  A  Lilliefor's 
goodness-of-f i t  test  can  be  used  to  check  the  assumption 
that  the  outputs  are  normally  distributed  (3).  The  Lillie- 
far's  result  will  test  the  following  hypotheses: 

Ha:  The  distribution  of  the  outputs  is  normal 

H*:  The  distribution  is  not  normal 

If  the  alternate  hypothesis  (H„)  is  indicated,  a  nonparamet- 
ric  test  can  be  used.  If  the  test  fails  to  reject  the  null 
hypothesis  (H0) ,  a  parametric  test  shpuld  be  used. 

Sel ecti on  of  a  Speci f i c  Stati st i cal  Test .  If  the 


decision  is  mads  to  use  a  parametric  comparison  test,  paired 
difference  confidence  intervals  can  be  used  (18:361-366). 

If  a  nonparametr i c  test  is  indicated,  the  Wilcoxon  signed 
rank  test  can  be  used  (18:682-685). 

Stati sti cal  T est  Procedure.  Regardless  of  the  specific 
test  selected,  the  general  procedure  for  the  tests  is  the 
same.  Each  model  will  be  run  a  number  of  times  with  the 
same  input  data.  The  output  data  from  one  model  will  be 
paired  with  corresponding  output  data  from  the  other,  and 
comparisons  made  using  the  appropriate  test.  The  exact 
wording  of  the  hypotheses  will  vary  depending  an  the  type  af 
test  performed. 

Paired  Pi f f erence  Confidence  Intervals.  Far  the 
paired  difference  confidence  interval,  a  point  estimate  of 
the  mean  population  difference  is  derived  by  averaging  the 
paired  differences  of  the  multiple  of  the  model  with  the 
same  TD5R  (2:453).  By  examining  the  confidence  intervals 
calculated  from  the  paired  differences,  the  hypotheses  be 
tested,  with  same  stipulated  level  of  confidence  (1-a).  In 
this  case,  the  hypotheses  will  be  stated  as: 

H0:  there  is  no  discernable  difference 

between  means 

H^:  there  is  sufficient  difference  between 

means  to  indicate  a  likely  true  differ¬ 
ence  between  papulations 

Wi 1  cox  on  Si gned  Rank  Test .  For  the  Wilccxon 
signed  rank  test  the  output  data  from  the  multiple  runs  can 
be  used  without  transf ormaticn.  Again  the  hypotheses  is  to 


be  tested  with  some  stated  1-evel  of  a.  The  hypotheses  will 
be  stated  as: 


Ha:  the  distributions  of  the  outputs  are  the 

same 

H*:  the  distributions  o-f  the  outputs  are 

different 

Sel ecti on  of  the  Confidence  Level .  Whichever  procedure 
is  used,  the  desired  confidence  level  for  the  tests  has  to 
be  stated  in  advance.  In  this  study,  the  cveral 1  hypotheses 
are  tested  by  the  individual  comparisons  of  numerous  output. 
In  order  for  the  confidence  in  the  overall  hypothesis  to  be 
as  high  as  desired,  the  confidence  level  of  each  of  the 
individual  comparisons  must  be  reduced  (3).  For  this  rea¬ 
son,  Bcnferroni's  method  is  used  to  calculate  a  value  of  a, 
for  the  individual  comparisons,  that  will  result  in  an  over — 
all  confidence  in  the  test  of  hypothesis  that  is  at  the 
desired  level  (3).  For  example,  if  an  overall  confidence 
level  of  907.  is  desired  for  conclusions  drawn  about  bath 
sortie  production  and  manpower  outputs,  at  all  3  target 
sortie  rates,  the  confidence  level  for  the  individual  com¬ 
parisons  (within  cells)  must  be  higher  than  907..  Bonfer — 
rani's  Method  indicates  the  appropriate  confidence  level  for 
the  individual  tests  (3). 

Categorization  of  Results 

Results  can  be  in  4  different  categories: 

1.  All  of  the  individual  tests  support  the 
alternate  hypothesis  (H.). 

2.  Most  of  the  individual  tests  support  the 
alternate  hypothesis  (H^.)  . 


Most  of  the  individual  tests  -fail  to  reject 
the  null  hypothesis  <H«). 


4.  All  o-f  the  individual  tests  -fail  to  reject  the 
null  hypothesis  (H^) . 

Eecause  o-f  the  nature  o-f  the  study,  inconclusive  results 
(categories  2  and  3)  are  likely. 

Deci sicn  Rules 

If  the  results  of  the  tests  fall  into  categories  1  or 
4,  the  conclusion  o-f  the  study  is  strai ghtf orward .  If  the 

results  all  refute  the  hypothesis  that  the  results  of  the 
simulations  are  the  same  (category  i),  the  conclusion  would 
be  that  the  outputs  of  the  two  models  are  different.  On  the 
other  hand,  if  no  result  refutes  the  hypothesis  of  equality 
(category  4),  then  a  conclusion  that  the  two  models  give  the 
same  outputs  is  warranted.  However,  if  the  results  fall 
into  categories  2  or  3,  no  conclusive  statements  can  be 
made.  Nonetheless,  any  tendency  noted  is  still  of  value  to 
the  purpose  of  this  study:  to  demonstrate  that  TSAR  either 
is  or  isn't  a  viable  alternative  to  LCCM  for  aircraft  main¬ 
tenance  requirements  forecasting.  Ultimately,  conclusions 
drawn  in  category  2  or  3  situations  will  rely  on  the 
subjective  evaluation  of  the  researcher. 
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HI.  Findings  and  Analysis 


Description  of  the  Actual  Experiment 


The  actual  comparison  of  TSAR  to  LCQM  included  several 
steps.  First,  the  databases  (inputs)  for  the  models  had  to 
be  made  as  compatible  as  possible  <5;  20).  Then  the  data¬ 
bases  were  run  against  their  respective  models,  with  target 
daily  sortie  rates  varied  from  1.0  to  2.0  to  3.0.  Finally, 
the  results  from  the  simulation  runs  had  tD  be  tabulated  and 
analyzed. 

Making  the  Databases  Compatible.  The  experiment  was 
hampered  by  the  lack  of  truly  compatible  databases.  TSAR  is 
constructed  as  a  wartime-only  model.  It  is  setup  to  handle 
a  two-shift  maintenance  operation  only  (as  opposed  to  the 
three  shifts  commonly  employed  in  peacetime  maintenance 
organizations),  and  there  are  other  features  of  the  model 
that  are  most  compatible  to  simulating  maintenance  in  a  wai — 
time  environment  (15).  Unfortunately,  the  only  LCQM  data¬ 
base  available  for  the  experiment  was  developed  for  simulat¬ 
ing  peacetime  maintenance  practices  (5).  Because  of  this 
inherent  difference  in  the  conceptual  design  of  the  data¬ 
bases,  numerous  changes  had  to  be  made  in  order  to  make  them 
as  compatible  as  possible.  In  keeping  with  the  premise  of 
testing  TSAR  while  holding  LCQM  as  a  standard,  the  bulk  of 
the  changes  were  made  to  the  TSAR  database.  Nonetheless, 
there  were  some  changes  made  to  the  LCOM  database  in  order 
to  maximize  compatibility. 


Changes  to  the  Databases.  Early  test  runs  of  TSAR 
clearly  showed  the  need  -for  changes  -for  the  sake  of  compati¬ 
bility.  The  TSAR  simulation  gave  results  indicating  much 
higher  achieved  sortie  rates  and  lower  man-hour  usage  than 
did  LCQM  runs  using  the  same  flying  schedule.  This  was  an 
expected  outcome  because  of  the  assumptions  TSAR  makes  about 
the  ability  of  the  maintenance  organisation  to  expedite  work 
under  wartime  conditions  <8;  9).  In  order  to  make  TSAR  as 
compatible  as  possible  with  LCOM  the  general  changes  listed 
in  table  I  were  made  to  the  databases.  See  appendix  A  for 
specific  variable  value  changes.  Adjustments  to  the  task 
times  and  the  pre-  and  post-sortie  delay  times  were  arrived 
at  through  analysis  of  the  flow  of  aircraft  through  the 
simulation,  excluding  unscheduled  maintenance.  Changes  were 
made  until  the  sortie  cycle  times  (the  time  from  the  begin¬ 
ning  of  a  sortie  until  the  aircraft  is  ready  for  a  subse¬ 
quent  sortie),  excluding  unscheduled  maintenance,  were 
'-cughly  the  same  between  LCOM  and  TSAR.  See  figure  3.  LCGM 
reconf i gurat i on  cutoff  times  were  reduced  to  zero  to  prevent 
the  automatic  loss  of  sorties  due  to  the  close  scheduling  of 
the  aircraft  missions  in  the  flying  schedule.  Some  of  the 
changes  to  the  databases  (e.g.  the  exclusion  of  cannibaliza¬ 
tion  and  phase  maintenance)  were  done  to  reduce  potential 
sources  of  variability  in  the  experiment,  as  opposed  to 
being  done  purely  to  maximize  compatibility. 

□ther  changes  to  TSAR  directly  reflect  the  differences 
between  maintenance  philosophies  for  wartime  and  aeaceti.ua 
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TABLE  I 


General  Changes  Made  to  Databases 


Model 


Change  <5;  20) 


TSAR 


Remove  resource  (manpower,  spare  part) 
constrai nts 


LCOM 


Remove  resource  (spare  part)  constraints 


TSAR  Remove  task  time  modifiers  (set  task  times 

to  100%  of  LCOM  standards) 


TSAR 


TSAR 


Adjust  pre-  and  post-sortie  delay  times 
Remove  hot-pit  refueling  option 


TSAR  Modify  task  structure  to  do  pre-  and  post- 

sortie  maintenance  tasks  in  series  versus 
parallel 


TSAR 


Remove  cannibalization  of  spare  parts  and 
SRUs 


TSAR 


TSAR 


Remove  option  to  defer  maintenance 
Delete  phase  maintenance  requirements 


LCOM 


Delete  reconf iguration  cutoff  times 


maintenance.  Examples  are  the  hot-pit  refueling  and  paral¬ 
lel  maintenance  tasks.  In  wartime,  these  tasks  are  often 
completed  in  those  ways  to  minimize  sortie  cycle  time  and 
thus  maximize  sortie  rate.  In  peacetime,  the  tasks  are 
normally  done  sequentially  (15). 

Flying  Schedul ss.  TSAR  and  LCOM  use  different  methods 
of  implementing  sortie  demands.  TSAR  uses  a  special  card 
type  <CT)  (CT  #50)  within  the  main  database,  for  submittal 
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Figure  4.  Flying  Schedules  far  1.0,  2.0  and  3.0  TDSRs 


of  the  desired  flying  schedule  (9:163-167).  LCOM  also  uses 
a  unique  farm  (farm  #20)  far  sortie  demands,  but  these  farms 
are  accumulated  in  a  separate  file  from  the  main  database, 
known  as  an  exogenous  file  (6:  Sec  3,  41-46).  Nonetheless, 
exact  duplication  af  flying  schedules  was  easily  attainable. 
Figure  4  graphically  displays  the  3  flying  schedules,  corre¬ 
sponding  to  the  1.0,  2.0  and  3.0  target  daily  sortie  rates 
(TDSR) ,  used  in  this  study.  See  appendix  A  for  the  detailed 
flying  schedules. 

Running  the  Nodal s.  Ideally,  the  models  would  have 
been  run  on  the  same  computer  system  to  alleviate  any  possi¬ 
ble  variance  stemming  from  the  operating  characteristics  of 
different  systems.  However,  this  was  impossible  since  no 
system  was  available  to  the  researcher  that  had  both  models 
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installed.  LCOM  was  run  an  a  NAS  7000  operating  system,  an 
IBM  compatible  mainframe  computer,  while  TSAR  runs  were  done 
or.  a  VAX  11/735  mi ni -computer .  Five  runs  of  thirty  days 
were  made  of  each  model  at  each  TDSR  to  get  the  needed  data 
for  use  in  the  comparisons.  Five  runs  was  chosen  as  a 
trade-off  between  the  need  for  accuracy,  which  would  have 
dictated  more  runs,  and  the  need  to  conserve  time  and  com¬ 
puter  resources,  which  demanded  that  the  number  of  runs  be 
kept  to  a  minimum. 

Sel ecti on  of  a  Confidence  Level .  As  described  in 
Chapter  2,  Bonferrcni's  method  was  used,  in  advance,  to 
determine  the  value  of  at  that  was  needed  far  individual  runs 


fj* 

1 


s 

$ 

& 

ft 


to  yield  the  desired  overall  level  of  confidence  for  the 
combination  of  all  comparisons.  A  desired  overall  confi¬ 
dence  level  of  ?07.  was  selected  for  the  comparison  of  TSAR 
and  LCQM.  Since  there  were  6  individual  comparisons  to  be 
node  (sorties  completed  and  man-hours  used,  at  each  of  3 
separate  TDSRs)  Bcnferroni's  method  yielded  a  confidence 
level  of  98.337.  (c<  =  .0167)  necessary  for  individual  compar¬ 
isons  (appendix  C) .  This  gave  a  two-tailed  t-value  of  3.966 
(4  degrees  of  freedom)  for  use  in  the  paired  difference  con¬ 
fidence  intervals  (appendix  C) . 


Results  of  Run  s 

The  results  of  the  38  runs  are  listed  in  appendix  B. 
The  first  task  in  processing  the  raw  data  was  tc  determire 


the  parametric  equation  assumption  of  normalcy  was  valid. 
App 1 i c  at i cn  of  Li  1 1 i ef  or ' s  Test.  As  explained  in 
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Chapter 


2, 


a  Lillie-far's  test  was  used  to  test  the  assump¬ 


tion  that  the  data  to  be  used  in  the  comparisons  came  -from 
normal  distributions  (3) .  The  test  was  applied  to  all  12 
groups  of  sample  data  using  a  desired  confidence  level  of 
99"/.  in  rejecting  the  assumption  of  normalcy.  The  confidence 
level  was  set  intentionally  high  because,  as  Law  and  Kelton 
note,  the  act  of  subtraction  in  constructing  a  paired  dif¬ 
ference  confidence  interval  helps  to  overcome  skewness  in 
the  distributions  of  the  random  variables  (16:319).  There¬ 
fore,  it  was  felt  that  a  good  deal  of  leniency  could  be 
shown  in  the  assumption  of  normalcy,  without  unduly  biasing 
the  results.  With  these  parameters,  the  Lillifor's  test 
failed  to  reject  the  hypothesis  that  the  samples  were  drawn 
fr-om  a  normal  distribution,  in  all  12  groups.  This  indi¬ 
cated  that  the  use  of  parametric  statistics  was  justified, 
and  therefore  the  use  of  a  paired  difference  confidence 
interval  was  appropriate. 

Paired  Pi f f erence  Conf i dence  Intervals.  Corresponding 
groups  cf  data  were  matched  and  individual  samples  paired  at 
random.  From  the  resulting  5  differences,  a  mean  and  stand¬ 
ard  deviation  were  calculated.  Using  this  data,  and  the  t- 
value  derived  from  the  application  of  Bonferroni  's  method, 
98.33"/.  confidence  intervals  were  constructed  for  each  of  the 
comparisons  (appendix  C) .  The  confidence  intervals  are 
shown  in  table  II. 
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Confidence  Intervals  -for  Paired  Differences  (LCOM  -  TSAR) 


Output 


Confidence  Interval 


Sorties 


1.59 


Man-Hours 


6128.3  ±  909.38 


Sorti es 


-72.2  ± 


26.  17 


Man-Hours 


12379.0  ±  1866.03 


Sorties 


-62.6  £  37.93 


Man-Hours 


13543.4  ±  1907.92 


Statistical  Analysis  of  Results 


As  outlined  in  Chapter  2,  the  hypotheses  tested  with  a 


paired  difference  confidence  interval  are  stated  as: 


H0:  there  is  no  discernable  difference 

between  means 


H_:  there  is  sufficient  difference  between 

means  to  indicate  a  likely  true  differ¬ 
ence  between  papulations 


In  tasting  these  hypotheses  with  a  confidence  interval,  the 


kev  is  whether  or  not  the  confidence  interval  contains  zero. 


If  so,  the  confidence  interval  indicates  no  reason  to  ""eject 


H„.  If  the  confidence  interval  does  not  contain  :sro,  is 


rejected  and  H..  accepted  <1:454).  Applying  these  rules  to 


the  confidence  intervals  in  table  II.  it  was  found  that  in 


all  cases,  save  one,  the  con-fidence  interval  indicated  suf 
ficient  differences  in  means  to  indicate  a  likely  true  dif 
ference  between  populations.  The  lone  exception  was  the 
mean  difference  in  sortie  rate  for  TDSR  =  1. 

Analysis  of  Resul ts  i n  Terms  of  Experimental  Hypotheses 
The  experimental  hypotheses  for  this  study  were: 

He,:  The  outputs  of  the  models  are  the  same 

H„:  The  outputs  of  the  models  are  different 
By  applying  the  categori zati on  of  results  listed  in  Chapter 
II,  a  category  2  situation  is  apparent:  most  of  the  indi¬ 
vidual  tests  support  the  alternate  hypothesis  (H-) .  As 
stated  in  the  decision  rules  of  Chapter  II,  this  is  not  a 
clear  cut  endorsement  of  either  hypothesis.  Therefore,  con¬ 
clusions  drawn  rely  on  the  judgement  of  the  researcher.  In 
this  case,  it  is  safe  to  say  that  the  results  of  the  paired 
difference  confidence  intervals  strongly  indicate  a  signifi¬ 
cant  difference  in  the  outputs  of  TSAR  when  compared  to  LCOM 
for  the  databases  used  in  this  study.  Figure  5  displays  a 
comparison  between  the  two  models  of  sortie  production  ver — 
sus  the  number  of  sorties  tasked  in  the  flying  schedules. 

It  gives  evidence  as  to  why  sortie  production  at  TDSR  1  is 
correlated  between  the  models.  Both  models  simulate  the 
completion  of  virtually  100%  of  the  sorties  tasked.  In 
other  words,  the  sortie  production  numbers  would  have  to  be 
the  same  simply  because  they  cannot  exceed  the  number  of 
sorties  tasked  in  the  flying  schedule.  Further  support  for 


the  alternate  hypothesis  (H„)  is  given  by  the  large 
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Figure  5.  Comparison  of  Mean  Achieved  Sorties  to  Tasked 
Sortie  Levels  at  Various  TDSR 

discrepancy  in  man-hour  outputs  between  the  two  models. 
Figure  6  clearly  shows  that  the  numbers  are  unmistakably 
different,  visually,  as  well  as  statistically. 

Pi  scussi  on  o-f  Resul  ts 

The  large  differences  between  the  man-hours  used  in  the 
two  models  seems  to  indicate  that  there  is  a  major  discrep¬ 
ancy  between  the  setup  of  the  databases.  Despite  the  re¬ 
searcher's  efforts  to  find  and  correct  differences  in  under¬ 
lying  assumptions,  it  seems  likely  that  such  is  the  source 
of  the  discrepancy.  Evidence  to  this  effect  can  be  found 
(figure  7)  by  examining  the  number  of  man-hours  used  as  com¬ 
pared  to  the  number  of  sorties  flown.  Note  that  the  lines 

73 


■  V  V  V  V  v  ,*  .... 

AtthblAfcA tot  mhiiuUd  iU 


MRINTENRNCE  MAN-HOURS 


Figure  6.  Comparison  o-f  Man-Hour  Outputs 


confidence.  The  very  small  range  of  this  confidence  inter¬ 
val  lends  credibility  to  a  belief  that  there  may  be  a  con¬ 
stant  or  nearly  constant  factor  difference  between  LCQM  and 
TSAR.  The  factors  that  account  for  the  approximately  9.5 
man-hours  per  sortie  difference  in  these  figures  may  be  the 
only  cause  of  the  apparent  difference  between  LCQM  and  TSAR. 

Findings  Relative  to  Research  Objectives 

In  addition  to  the  statistical  findings  in  support  of 
the  experimental  hypotheses,  the  study  was  undetaken  to  ex¬ 
plore  the  three  research  questions  listed  in  Chapter  1. 

Differences  Between  Networks.  The  first  research  ques¬ 
tion  was:  What  differences  exist  between  LCOM  and  TSAR  simu¬ 
lations?  The  answer  to  this  question  is  quite  lengthy. 
Despite  the  fact  that  the  maintenance  networks  in  the  TSAR 
database  were  adapted  from  those  in  LCOM,  gross  differences 
in  the  models  exist.  While  by  no  means  an  exhaustive  list, 
the  following  are  some  of  the  differences  that  may  be  of 
consequence  to  potential  TSAR  users. 

Task  Definitions.  The  F-16  TSAR  database  used  in 
this  study  <19),  appears  to  combine  maintenance  tasks  de¬ 
picted  as  separate  tasks  in  the  LCQM  simulation  of  the  same 
series  of  tasks.  As  a  result,  task  times,  and  task  proba¬ 
bilities  for  unscheduled  maintenance  tasks,  must  be  com¬ 
bined.  This  makes  it  exceedingly  difficult  for  the  novice 
TSAR  user  to  make  side-by-side  comparisons  of  LCQM  and  TSAR 
maintenance  networks  to  insure  compatibility.  Spot  checks 
of  TSAR  networks  by  the  researcher  seemed  to  indicate  that 


the  net  effect  of  the  two  methods  is  the  same;  both  accomp¬ 


lish  the  same  tasks  approx i matel y  the  same  percentage  of  the 
time,  with  approximately  the  same  expenditures  o-f  resources 
and  time.  Further,  no  readily  apparent  reason  could  be 
found  for  TSAR's  compression  of  tasks.  Ultimately,  the 
users  who  want  to  dig  into  the  networks  and  see  for  them¬ 
selves  that  a  network  is  an  accurate  depiction  of  a  real- 
life  occurrence,  will  feel  more  at  ease  with  LCQM  databases, 
ar  TSAR  networks  that  are  configured  like  LOOM  databases. 

The  networks  are  simply  more  detailed  and  have  a  better 
one-for-cne  correspondence  with  actual  maintenance  activity 
in  LC3M.  For  users  that  don't  want  or  need  to  know  the 
exact  details  of  the  how  the  simulation  matches  with  actual 
maintenance  practices,  this  observation  is  of  no  concern. 

In  fact,  for  the  more  casual  user,  TSAR  has  the  advantage  in 
having  much  fewer  total  tasks  with  which  to  have  to  deal. 

It  seems  safe  to  speculate  that  the  lower  level  of  detail  in 
TSAR  helps  contribute  to  its  faster  execution  speed  and  its 
ability  to  model  a  much  broder  range  of  airbase  and  theater 
activities.  This  may  be  the  reason  far  the  task  combina¬ 
tions  noted.  It  is  passible  that  TSAR  networks  could  be 
constructed  so  as  to  more  nearly  match  the  LCOM  format, 
especially  for  relatively  simple  databases. 

Data  Entry .  As  a  result  of  the  much  broader  set 
cf  options  available  to  the  TSAR  user,  the  T5AR  database 
requires  a  much  larger  number  of  different  input  variacles. 
Whereas  LCCM  uses  13  different  forms  for  data,  entry,  plus  a 


number  of  change  card  options  that  generally  control  the 
operation  of  a  single  switch  or  variable  <6:  Sec  4,  2),  TSAR 
has  1(37  different  card  types  <9:228-236),  although  it  is 
unlikely  that  all  would  be  used  in  a  single  database.  In 
the  F-16  databases  used  in  this  study  there  were  9  LOOM 
forms  used  (5)  and  62  TSAR  card  types  used  (20).  The  dif¬ 
ferences  are  not  as  great  as  these  number  would  lead  to 
believe,  however.  In  both  cases,  the  databases  contained  a 
number  of  options  that  were  not  used  in  this  particular 
study,  but  TSAR  has  by  far  a  larger  number  of  cards,  which 
are  dedicated  to  entering  data  necessary  to  simulate  the 
entire  airbase  and  theater  environment,  but  weren't  neces¬ 
sary  to  this  particular  study.  In  addition,  LCOM  utilizes  a 
change  card  file  <6:  Sec  4,  2-27)  that  initiates  a  number  of 
the  options  available  to  TSAR  as  a  separate  card  type.  Nine 
change  cards  were  used  in  the  LCOM  runs  done  for  this  study, 
roughly  equivalent  to  the  options  available  on  1  TSAR  c_..  a 
type.  Despite  these  ameliorating  factors,  TSAR  databases 
are  much  more  complex  in  their  structure.  For  a  user  that 
doesn't  need  the  great  flexibility  advantage  that  TSAR  of¬ 
fers,  a  LCOM  database  is  easier  to  "find  your  way  around". 
The  sheer  number  of  options  available  to  the  TSAR  user  can 
be  bewildering  and  make  it  difficult  to  insure  that  the  de¬ 
sired  activities  occur  while  these  not  wanted  are  sup¬ 
pressed  . 

LCOM  has  the  disadvantage  in  initial  simulation  prep¬ 
aration  cf  having  to  use  at  least  3  different  files 
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(initialization,  exogenous  and  change)  to  accomplish  data 


entry  (6:  Sec  3,  1-3).  All  TSAR  data  entry  can  be  done  in 
the  same  file,  unless  attack  simulation  is  required.  In 
soma  cases  the  use  of  exogenous  and  change  files  can  simpli¬ 
fy  the  use  of  LCQM,  though.  This  is  especially  true  when 
different  flying  schedules  or  input  changes  are  used  against 
the  same  basic  database.  Several  different  exogenous  and/or 
change  files  can  be  produced  and  used  to  alter  the  simula¬ 
tion,  without  having  to  edit  the  basic  database.  For  in¬ 
stance,  in  this  study,  three  different  flying  schedules  were 
used.  With  LOOM,  the  flying  schedules  were  prepared  and 
submitted  to  three  separate  exogenous  files.  Changes  to  the 
maintenance  networks  had  no  effect  on  these  files.  With 
TSAR  on  a  mi ni -computer ,  using  three  different  flying  sched¬ 
ules  required  the  use  of  three  separate  databases,  or  the 
editing  of  the  database  each  time  a  different  flying  sched¬ 
ule  was  needed.  If  three  databases  were  used,  changes  to 
the  maintenance  networks  had  to  be  made  to  all  three  data¬ 
bases  in  order  to  keep  them  current.  On  a  more  capable 
operating  system,  this  disadvantage  would  largely  or  com¬ 
pletely  disappear  (15). 


Taking  Runs.  The  ease  of  making  runs  varies  bet¬ 
ween  the  two  models.  LCOM  has  the  advantage  of  having  exog¬ 
enous  and  change  files,  discussed  in  the  preceding  section. 
However,  when  doing  multiple  runs  with  the  same  database, 


"3AR  has  clear  advantages.  With  a  quick  change  to  card  type 
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done  <9:181.  TSAR  automatically  changes  random  number 
streams  at  the  beginning  of  each  run,  i-f  desired,  to  produce 
a  distribution  o-f  outputs.  LCOM,  on  the  other  hand,  re¬ 
quires  the  user  to  submit  runs  singly.  If  changes  in  random 
number  streams  are  necessary,  initialization  seeds  must  be 
specified,  in  the  change  card  file,  for  each  stream  to  be 
changed  (6:  Sec  4,  16).  Another  advantage  for  TSAR  is  its 
faster  execution  time.  Since  the  two  models  were  run  on 
different  systems,  with  different  loading  factors,  direct 
comparison  should  not  be  interpreted  as  an  accurate  reflec¬ 
tion  of  exact  differences  in  model  execution  times.  How¬ 
ever,  the  average  time  for  the  15  TSAR  runs  done  for  this 
study  was  1.1979  minutes,  while  the  average  for  the  15  LCOM 
runs  was  12.642  minutes,  more  than  a  factor  of  10  (appendix 
A).  It  should  be  remembered  that  TSAR  runs  were  done  on  the 
less  capable  of  the  two  systems,  therefore  the  actual,  in¬ 
herent  difference  in  execution  speed  between  LCOM  and  TSAR 
is  likely  even  larger  than  that  indicated.  Since  LCOM  and 
TSAR  are  both  normally  done  as  batch  runs,  that  is  the  runs 
are  submitted  and  the  computer  system  holds  them  until  sys¬ 
tem  loading  is  favorable,  the  actual  difference  in  turn¬ 
around  time  between  LCOM  and  TSAR  can  stretch  into  hours,  in 
the  researcher's  experience. 

Outputs.  LCOM  and  TSAR  take  very  different  ap¬ 
proaches  to  presentation  of  outputs  and  there  are  many 
differences  between  the  results  achieved.  Botn  have  a  great 
number  of  options  regarding  how  much  detail  the  user  cares 


to  get  from  the  simulation.  In  addition  to  the  outputs 
available  from  the  actual  model  run,  LCQM  offers  a  number  of 
post-processor  options  that  can  further  increase  the  variety 
of  outputs  the  user  can  obtain  <6:  Sec  5,  3-4) .  At  the  time 
of  the  study,  TSAR  did  not  have  a  post-processor  available, 
although  provisions  have  bean  made  in  the  coding  of  the 
model  to  accomodate  a  past-processor .  Since  there  are  so 
many  options  at  the  disposal  of  the  user  of  either  LCOM  or 
TSAR,  it  is  infeasible  to  address  even  a  fraction  of  the 
possible  advantages  of  one  over  the  other.  Rather,  the 
researcher  feels  two  features  are  of  special  note,  bath  with 
TSAR.  On  the  positive  side,  TSAR,  by  virtue  of  its  capabil¬ 
ity  to  do  multiple  runs,  presents  multiple  trial  results  for 
an  entire  series  of  runs.  For  instance,  in  this  study  five 
runs  were  dona  with  each  flying  schedule.  TSAR  presented  the 
average  number  of  sorties  flown  as  a  part  of  the  multiple 
trial  results,  eliminating  the  need  to  search  the  output  for 
individual  data  elements  in  order  to  make  quick  checks  of 
run  success.  Cn  the  other  hand,  although  man-hours  are 
clearly  tracked,  TSAR  has  a  deficiency  in  its  handling  of 
manpower  utilisation  outputs,  at  least  from  the  standpoint 
of  a  manpower  analyst.  Man-hours  used  are  reported  by  the 
shop  to  which  the  personnel  are  assigned.  This  is  less  de¬ 
sirable  than  the  LCQM  procedure,  in  which  maintenance  man- 
hours  are  reported  by  AFSC,  since  the  manpower  analyst  is 
-oread  to  convert  from  shop  number  to  AFSC.  Since  TSAR 
provides  for  only  30  shops  <8:37)  a  maximum  of  30  AFSCs  can 


be  tracked  separately 


That  in  itself  may  be  a  relatively 


trivial  problem,  especially  in  light  of  Rivet  Workforce 
efforts  to  reduce  the  number  of  maintenance  AFSCs  (13)  but 
the  problem  is  compounded  because  there  is  not  a  one-for-one 
correspondence  between  shop  type  and  personnel  type,  in  all 
cases.  For  instance,  in  the  F-16  TSAR  database  used  for 
this  study,  shop  #30  has  both  461X0  and  423X3  AFSC  personnel 
assigned  (19:  Sec  III,  37).  Further  confusion  stems  from 
the  accounting  of  man-hours.  lian-hours  are  accumulated  in 
the  shops  to  which  the  personnel  are  assigned,  even  though 
they  may  be  performing  a  task  they  are  cross-trained  for,  in 
another  shop  (15).  These  factors  make  TSAR  difficult  to  use 
for  manpower  assessments.  Therefore,  to  be  of  the  most  ben¬ 
efit  to  the  manpower  community,  the  accumulation  of  man-hour 
data  within  TSAR  would  have  to  be  modified. 

Changes  Necessary  to  Make  TSAR  Compatible  to  LCQM.  The 
second  research  question  was:  In  what  ways  must  TSAR  be 
manipulated  to  give  acceptable  forecasts?  This  question 
cannot  be  fully  addressed  because  the  study  failed  to  fully 
achieve  standardi zat i on  between  the  databases.  Nonetheless, 
the  final  database  used  in  the  study  gave  outputs  that  were 
much  closer  to  LCQM  outputs  than  did  the  original,  unchanged 
database  documented  by  Orlando  Technology,  Inc  (19).  Gen¬ 
eral  changes  made  are  listed  in  table  I,  while  specific 
changes  are  listed  in  appendix  A.  As  a  caution  to  new  TSAR 
users,  the  collateral  effects  of  a  number  of  different  vari¬ 
able  inputs,  scattered  over  a  number  of  different  card 


types,  go  into  the  operation  of  sometimes  seemingly 


QC 
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strai ght-f orward  tasks.  An  example  is  the  loading  of  muni¬ 
tions  onto  the  aircraft.  TSAR  di f f erenti ates  between  basic 
munitions  and  mission-specific  munitions  and  each  is  handled 
differently.  Basic  munitions  are  specified  on  card  type 
(CT)  #15/1).  Basic  munition  loading  is  specified  as  a  task 
or  tasks  in  the  on-equipment  maintenance  networks  (CT  #5s) . 
The  probability  that  a  particular  sortie  will  require  load¬ 
ing  of  basic  munitions  (after  the  initial  conf i gurati on  of 
the  aircraft)  is  set  by  the  probability  of  basic  munitions 
retention  on  a  different  card  type  (CT  #16/1),  as  opposed  to 
the  usual  method  of  controlling  an-equipment  maintenance  by 
specifying  network  entry  probabi 1 i ti es  (CT  #7).  Task  times 
and  resource  requirements  for  basic  munitions  loading  are 
contained  in  the  CT  #5.  Mission  specific  munitions  are 
specified  through  a  different  process.  The  mission  type 
required  is  specified  in  the  flying  schedule  (CT  #50).  From 
that  requirement,  the  type  of  standard  combat  load  (SCL) 
required  is  found  (CT  #12).  With  the  SCL  known,  the  proba¬ 
bility  that  the  SCL  has  been  retained  is  found  on  CT  #16/1, 
separate  from  the  probability  of  retention  of  basic  muni¬ 
tions.  If  necessary,  the  specific  munitions  to  be  loaded, 
the  resources  needed  and  the  task  times  are  found  (CT  #13). 
In  addition  to  the  card  types  that  directly  relate  to  muni¬ 
tions  leading,  TSAR  has  a  number  of  user  controllable  op¬ 
tions  ttiat  can  modify  the  processing  of  the  tasks.  For  in¬ 
stance,  the  database  may  have  a  task  time  modifie-  specified 


(CT  #17/2),  which  can  reduce  or  extend  the  times  shown  with 
the  individual  task  <S) .  As  this  example  illustrates, 
tracking  down  the  source  of  a  discrepancy  within  TSAR  is  no 
trivial  task.  Thorough  review  of  the  database  is  essential 
Shortcomings  in  TSAR.  The  third  research  question  was* 
If  TSAR  fails  to  provide  acceptable  forecasts,  are  there 
apparent  shortcomings  in  TSAR  that  can  be  economically  over — 
came 7  Although  TSAR  failed  to  give  forecasts  compatible 
with  those  from  LCOM,  the  exact  reason  for  this  failure 
could  not  be  determined.  It  is  very  likely,  in  light  of  the 
close  tracking  of  the  normalized  man-hours  between  LCOM  and 
TSAR  (figure  8),  that  the  only  real  failure  was  the  failure 
of  the  researcher  to  uncover  the  factor  or  factors  that 
remain  in  disagreement  between  the  peacetime  LCOM  database 
and  the  wartime  TSAR  database.  However,  in  the  area  of 
outputs  there  are  useful  changes  that  could  be  made.  For 
instance,  the  accumulation  of  maintenance  man-hours  by  AFSC 
or  the  development  of  a  post-processor  that  would  do  so, 
would  greatly  aid  the  prospective  manpower  analyst-TSAR 
user.  Conceptually  this  not  a  difficult  problem.  In  addi¬ 
tion,  projects  are  currently  underway  to  produce  a  TSAR  pre¬ 
processor  (4).  The  pre-processor  would  take  a  mors  easily 
understood,  user-produced  database  and  convert  it  into  the 
proper  coding  for  TSAR  simulations  (4).  This  would  reduce 
or  eliminate  the  problem  of  TSAR's  complexity  for  initial 


data  entry  ( 15) , 


a  sorely  needed  improvement. 


IV.  Conclusions  and  Recommendation 


Si  cni  f  i  cance  of  Findinas 


The  findings  of  this  study  can  be  broken  into  two 
groups,  statistical  and  qualitative.  The  significance  of 
the  findings  varies  from  one  to  the  other. 

Si qni f i cance  of  Statistical  Findings.  The  validity  of 
the  statistical  findings  of  this  study  was  greatly  reduced, 
by  the  lack  of  databases  with  the  same  basic  assumptions. 

The  LCOM  database  used  was  based  an  peacetime  maintenance 
concepts  and  practices,  while  TSAR  databases  are,  by  defini¬ 
tion,  designed  with  wartime  maintenance  in  mind.  Given 
these  proolems,  along  with  the  great  complexity  of  the 
models,  it  is  not  surprising  that  the  outputs  of  the  models 
are  significantly  different.  These  figures  really  tell 
little  about  the  relative  merit  of  TSAR  in  relation  to  LCGM. 
If  any  conclusion  can  be  drawn,  it  is  that  TSAR's  capability 
is  hinted  at  by  the  nearness  to  which  its  man-hour  outputs 
parallel  those  of  LCGM  (figures  7  and  S) . 

Si gni t i cance  of  Qual i tat i ve  Findings.  The  experiences 
of  the  researcher  may  be  of  great  value  to  future  users  of 
TSAR,  or  to  those  who  contemplate  using  TSAR  in  place  of,  or 
as  a  compliment  to,  LCGM.  TSAR  is  a  relatively  new  model 
and  few  people  have  experience  in  running  both  TSAR  and 
LCGM,  especially  on  the  same  problem.  Thus  the  relative 
merits  and  advantages  of  one  model  over  the  other  are  not 


well -known,  aside  from  the  uni  cue  capabi 1 i tii 


'SAR  to 
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simulate  extended  airbase  and  theater  activities. 


Pr acti cal  Impl ications  of  F i ndi nqs 

The  adaptability  o-f  TSAR  to  peacetime  maintenance 
concepts  and  practices  has  not  been  shown  by  this  study. 
Therefore,  -further  study  is  warranted  before  concrete 
conclusions  can  be  drawn  about  the  use  of  TSAR  to  replace 
LCOM.  Because  of  this  lack  of  proof,  and  because  LCOM  is 
easier  to  use  in  many  ways  than  TSAR,  it  seems  prudent  to 
use  LCOM  if  projected  uses  are  within  the  bounds  of  LCOli's 
capabilities.  However,  there  is  a  growing  need  to  assess 
requirements  using  a  systems  approach.  This  implies  consi¬ 
dering  the  interaction  of  various  components  of  a  larger 
system  rather  than  looking  at  the  components  in  isolation. 
LCOM  is  recognized  as  doing  a  good  job  at  forecasting  air¬ 
craft  maintenance  requirements  at  a  single  base.  However, 
there  are  pressures  on  the  requirements  community  to  expand 
their  research  to  include  the  interactions  of  aircraft  main¬ 
tenance  with  other  airbase  activities,  enemy  attack  and 
theatre  operations.  These  needs  threaten  to  overload  the 
single  airbase,  maintenance-only  capabilities  cf  LOOM. 

Whan,  for  these  reasons,  an  analyst  is  driven  to  seek  more 
powerful  tools  than  LCOM,  TSAR  has  natural  attractions.  It 
is  at  that  point  that  the  qualitative  findings  of  this  stud- 
can  be  useful  in  the  assessment  of  TSARs  appl i cabi 1 i ty  and 
use.  It  should  be  remembered  that  TSAR  is  very  new  and  has 
lets  of  room  for  future  development.  Those  who  have  worked 
with  LCOM  for  a  number  of  years  can  attest  tc  the  fact  that 
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earlier  versions  of  LCOM  had  shortcomings  much  like  those 
identified  in  TSAR:  weak  output  and  complex  inputs  (4). 
These  problems  can  only  be  overcome  through  work  with  the 
model  to  identify  areas  for  improvement  and  continual 
development  to  fix  the  shortfalls. 

Recommendati ons  for  Fol low-on  Experiments 

This  study  can  be  thought  of  as  an  ice-breaker.  The 
experiences  of  the  researcher  bring  up  more  questions  than 
they  answer  and  point  the  way  towards  some  promising  avenues 
of  further  research.  But  a  few  basic  qualifiers  need  to  be 
addressed.  Part  of  the  difficulty  in  getting  good  answers 
in  this  study  was  the  complexity  of  the  models.  It  takes  a 
lot  of  time  and  effort  to  become  familiar  with  one,  let 
alone  two,  simulation  programs  of  this  magnitude.  Future 
researchers  into  the  differences  between  these  two  models 
would  be  well  served  to:  allocate  about  a  year's  time  to 
the  study,  or,  work  in  tandem  with  another  researcher  (with 
each  taking  prime  responsibility  for  learning  one  of  the  two 
models),  or  have  a  goad  working  knowledge  of  one  of  the 
models  prior  to  undertaking  the  study.  A  researcher  consi¬ 
dering  doing  a  study  of  this  type  should  be  comfortable  with 
computers,  enjoy  the  challenge  of  a  jigsaw  puzzle-tyoe  of 
problem  and  be  able  to  work  well  alone  or  with  one  or  two 
ether  researchers. 

Seme  avenues  that  appear  to  be  promising  for  future 
research  include  the  following: 


Replicate  this  studv ,  picking  up  wnere  the 


researcher  left  off,  and  continue  the  search  for  factors 
that  differ  between  LCQM  and  TSAR.  Comparisons  of  outputs 
other  than  sortie  production  and  man-hours  are  passible  and 
will  have  to  be  done  eventually. 

2.  Narrow  the  scope  of  the  study  and  look  at  some 
subset  of  the  models.  It  is  feasible  to  isolate  certain  of 
the  networks,  for  example,  and  run  the  models  with  just 
those  networks  active.  Another  possibility  is  to  adjust 
task  probabilities  so  that  only  scheduled  maintenance  and 
sortie  configuration  is  done. 

3.  Find  a  wartime  LCQM  database  and  run  it  in  compari¬ 
son  to  an  unaltered  TSAR  database.  Attempt  to  identify  haw 
the  databases  parallel  one  another  and  how  they  mutually 
differ  from  peacetime  databases. 

4.  Build  simple  TSAR  and  LCOM  databases  from  scratch 
and  run  in  comparison.  This  would  definitely  be  a  job  for 
two  people,  but  it's  one  way  of  assuring  compatibility  of 
assumpti ons. 


Appendix  A 


Model  Inputs 


TABLE  III 

Changes  Made  to  TSAR  Database,  Primary  Control  Cards 


Card  Type 

Variable 

Original  Value 
(19) 

New  Value 

1 

NTRIAL 

25 

2/1 

PRINT 

4 

0 

STATFQ 

10 

3 

CUMSTA 

1 

0 

2/2 

Sortie 

Demand 

-1 

0 

3/1 

POSTPN 

1 

0 

DOPHAS 

1 

0 

CANMQD 

0 

CANSRU 

1 

3 

3/2 

NOFUEL 

0 

1 

NEWDATA 

0000 

1603 

3/3 

RANDM 

1 

0 

FULL 

0 

1 

HIATUS 

14 

0 

NPART 

271 

387 

TABLE  IV 


Changes  Made  to  TSAR  Database,  Resource  Requirements  Data 


Card  Type 

Field  Name 

Old  Value 
<  19) 

New  Value 

5  #857 

TTU 

r> 

4 

5  #850 

TTU 

i 

n 

13  #1 

MEAN  TTU  #1 

7 

10 

15/1 

DELAY /PQST-FLT 

16 

90 

DELAY /PRE-FLT 

29 

49 

17/1 

AC  LOADS 

1 

REFILL  TIME 

125 

45 

17/2 

HURRY/QN  EQPT 

70 

100 

HURRY/PRE-FLT 

70 

1 00 

HURRY/OFF  EQPT 

70 

100 

HURRY/MUN  ASMB 

70 

100 

Table  V 

Changes  to  TSAR  Database,  Initial  Stacks  o-f  Base  Resources 


Table  VI 


i 

i 


Changes  to  TSAR  Database,  Comm.  Systems  and  Initialization 


Card  Type 

Field  Name 

□Id  Value 
(19) 

New  Value 

33 

CLASS 

(CARD  ADDED) 

3 

MEAN  TIME  (HRS) 

6 

CLASS 

2 

MEAN  TIME  (HRS) 

6 

41  #1 

#  ASSIGNED/MSN1 

0 

24 

#  ASSIGNED/MSN2 

48 

0 

#  ASSIGNED/MSN3 

24 

0 

Table  VII 

Changes  to  LCOM  Database  (5) 


Form  21: 

ACAP  to 

CAP  Cutoff 

Time  Changed  From  1.8  to  0.0 

Form  21: 

CAPCAP 

to  CAP  Cutoff  Time  Changed  From  2.7  to  0.0 

Change  Cards: 

VNAM 

IPAR 

FPAR 

1APAR  IPAR2  FPAR2 

CQNSUM 

3 

ACF 

KTIMSW 

18 

ACF 

GENRAT 

~T 

■— * 

ACF 

NCJCLWM 

RFREG! 

31 

STQRAC 

3 

ACF 

AUTH 

389 

9999 

AUTH 

399 

999 

STOP 

31.1 

' ISEED) 

Used  As  Necessary 

for  Streams  #2,3,6, 7 

Flying  schedules  -far  LCOli  and  TSAR  were  -functionally 
identical.  Each  schedule  called  for  the  same  pattern  of 
mission  demands  each  day  for  30  days.  The  LCOM  schedule 
called  for  the  flights  to  begin  an  Day  2  and  to  continue 
through  Day  31,  to  avoid  lasing  sorties  an  Day  1  due  to  the 
long  FRAG  time  used,  while  TSAR's  schedules  began  on  Day  1. 
The  FRAG  was  set  at  18  hours  prior  to  takeoff.  All  sorties 
flew  the  same  mission  profile,  and  all  missions  were  for  1.1 
hours  (constant).  Four  sorties  per  mission  were  requested, 
with  a  minimum  of  2  needed  to  avoid  mission  abort.  Missions 
could  takeoff  up  to  15  minutes  late.  Mission  takeoff  times 
are  shown  in  Table  VIII. 

Table  VIII 

Flying  Schedul es-Target  Daily  Sortie  Rate  (TDSR)  1.0/2. 0/3.0 


BsP 

1 

Mission  Time 

#  of  Msns 

#  of  Msns 

#  of  Msns 

Appendix  B.  Model  Outputs 

Table  IX 

Sortie  Production  and  Man-Hour  Outputs  far  LOOM  and  TSAP 


LOOM 


TSAR 


TDSR 

Run  # 

Sort i es 

Man-Hours 

Sorties 

Manhours 

1.0 

1 

720 

17932 

720 

12590 

*■> 

719 

18877 

720 

1 2930 

719 

18884 

720 

12290 

4 

719 

18961 

72  0 

12750 

5 

720 

19110 

719 

12560 

2.0 

1 

1323 

34301 

1394 

21500 

o 

1304 

32519 

1381 

21950 

1305 

35293 

1397 

22030 

•  4 

1321 

34580 

1372 

22270 

5 

1309 

34762 

1379 

21840 

3.0 

1 

1481 

37699 

1559 

24440 

1479 

39272 

1535 

24630 

3  1 

1477 

38275 

1516 

25980 

4 

1487 

38158 

1536 

23770 

5 

1478 

38123 

1569 

25240 

Table  X 

Simulation  Run  Execution  Times  (Seconds) 

Mcdel/TDSR  Run  #1  Run  #2  Run  #3  Run  #4  Run  #5 


LCCM/1 . 0 
LCGM/2.0 
LCOM/3.0 

TSAR/ 1.0 
TSAR/2.0 
TSAR/3.0 


307.2 
581 . 4 
720 

251. 16* 
373.99* 
452.98* 


459 

847.2 

1050.6 


426.  6 
990.  6 
1006.2 


595.2 
1616.4 
1003. 8 


342 

687.6 

741 


'Total  of  all  5  runs 


Eft 

3 

s88 

Appendix  C.  Calculations 


Paired  Dif f ersnce  T-Test 

The  paired  di f f erencs  t-test  was  used  to  compare  model 
outputs  by  forming  a  paired  difference  confidence  interval 
and  checking  to  see  if  the  interval  contained  zero  (18:362). 
The  general  form  of  the  equation  used  for  the  paired  differ — 
ence  confidence  interval  is: 


Xd 


So 


t  Ct  y  ~  - 


where 


r<o  =  the  mean  paired  differences 
tat.'-  =  value  at  desired  ct  level  with  (nD  -  1)  degrees 
of  freedom 

So  =  standard  deviation  of  paired  differences 
n0  =  number  of  paired  differences 


Scnf erroni ' s  Method 

In  order  to  insure  that  the  level  of  confidence  for  the 
entire  experiment  was  as  high  as  desired,  it  was  n'ecessary 
to  reduce  the  value  of  ct  for  the  individual  comparisons. 
Banfsrrani 's  method  was  used  to  calculate  the  level  of  at  for 
the  individual  comparisons  (3).  Bcnfsrroni's  equation  is: 


Ctcr 

P 


where 


at  =  error  associated  with  confidence  level  of  individ¬ 
ual  tests 

etc  =  error  associated  with  desired  overall  confidence 
p  =  number  of  individual  tests  to  be  performed 


Li  1 1  i  ef  or  '  s  Test  -for  Normal  i  ty 

One  a-f  the  assumptions  that  it  was  necessary  to  make  in 
the  use  of  the  paired  difference  t-test  was  that  the  rela¬ 
tive  frequency  distribution  of  the  population  cf  differences 
was  normal  (18:362).  A  Lilliefor's  test  was  used  to  test 
this  assumption  (3).  The  equation  for  the  Lilliefor’s  test 
statistic  is: 

D  =  sup  |Fa(;<)  -  S(x)| 

where 

sup  -  means  find  the  supremum  (maximum)  over  all  values 
of  x . 

F0(x)  =  the  value  of  the  hypothezied  cumulative  distribu¬ 
tion  function  (CDF)  at  x 
5(x)  =  the  value  of  the  obseved  sample  CDF  at  x 

The  value  of  D  derived  from  this  equation  was  compared  to  a 

critical  value  of  D  for  the  sample  size,  found  in  the  Lil¬ 


liefor's  table  (3). 
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